Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Current use of simulation for EPA assessment in emergency medicine.

OBJECTIVE: Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment. METHODS: A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views. RESULTS: A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment. CONCLUSIONS: Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

RéSUMé: OBJECTIF: Il y a environ cinq ans, les programmes de médecine d'urgence du Collège royal au Canada ont mis en place un paradigme basé sur les compétences et ont introduit l'utilisation d'activités professionnelles confiables (APC) pour l'évaluation des unités d'activité professionnelle afin d'évaluer les stagiaires. De nombreux programmes d'enseignement médical basés sur les compétences (CBME) prévoient l'évaluation des compétences par l'observation des APC. Bien que les APC soient fréquemment évaluées en milieu clinique, la simulation est également utilisée. Cette étude visait à caractériser l'utilisation de la simulation pour l'évaluation de l'APC. MéTHODES: Un guide d'entretien d'étude a été élaboré conjointement par tous les auteurs de l'étude et a suivi les meilleures pratiques en matière d'élaboration d'enquêtes. Un entretien national a été réalisé avec les directeurs de programmes ou les directeurs adjoints de tous les programmes de médecine d'urgence du Collège royal au Canada. Les entretiens ont été menés sur Microsoft Teams, enregistrés et transcrits à l'aide du service de transcription de Microsoft Teams. Les transcriptions des échantillons ont été analysées pour développer des thèmes. Les thèmes ont ensuite été revus par les co-auteurs pour s'assurer qu'ils étaient représentatifs des points de vue des participants. RéSULTATS: Un taux de réponse de 64,7 % a été obtenu. La simulation a été largement adoptée par les programmes de formation en médecine d'urgence. Toutes les personnes interrogées se sont montrées favorables à l'utilisation de la simulation pour l'évaluation de l'APE pour de nombreuses raisons. Cependant, les DP ont reconnu des limites et l'analyse thématique a révélé certains thèmes et tensions liés à l'utilisation de la simulation pour l'évaluation de l'APC. L'analyse thématique a révélé six thèmes majeurs : un appui généralisé à l'utilisation de la simulation pour l'évaluation de l'APC, inquiétudes concernant la possibilité que l'évaluation de l'APC devienne un exercice de type « case à cocher ¼, des obstacles logistiques limitant l'utilisation de la simulation pour l'évaluation de l'APC, les perceptions variées quant à l'authenticité de l'utilisation de la simulation pour l'évaluation de l'APC, le potentiel de la simulation pour l'évaluation de l'APC de compromettre la sécurité psychologique des apprenants, et des suggestions pour l'optimisation de l'utilisation de la simulation pour l'évaluation de l'APC. CONCLUSIONS: Nos résultats offrent un aperçu à d'autres programmes et spécialités sur la meilleure façon d'utiliser la simulation pour l'évaluation de l'APC. Les programmes devraient utiliser ces résultats lorsqu'ils envisagent d'utiliser la simulation pour l'évaluation de l'APC.

Implementing Competence Committees on a National Scale: Design and Lessons Learned.

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.

Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Coaching in Competence by Design: A New Model of Coaching in the Moment and Coaching Over Time to Support Large Scale Implementation.

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

The Impact of Just-in-Time Simulation Training for Healthcare Professionals on Learning and Performance Outcomes: A Systematic Review.

ABSTRACT: Although just-in-time training (JIT) is increasingly used in simulation-based health professions education, its impact on learning, performance, and patient outcomes remains uncertain. The aim of this study was to determine whether JIT simulation training leads to improved learning and performance outcomes. We included randomized or nonrandomized interventional studies assessing the impact of JIT simulation training (training conducted in temporal or spatial proximity to performance) on learning outcomes among health professionals (trainees or practitioners). Of 4077 citations screened, 28 studies were eligible for inclusion. Just-in-time training simulation training has been evaluated for a variety of medical, resuscitation, and surgical procedures. Most JIT simulation training occurred immediately before procedures and lasted between 5 and 30 minutes. Despite the very low certainty of evidence, this systematic review suggests JIT simulation training can improve learning and performance outcomes, in particular time to complete skills. There remains limited data on better patient outcomes and collateral educational effects.

Society for Simulation in Healthcare Guidelines for Simulation Training.

BACKGROUND: Simulation has become a staple in the training of healthcare professionals with accumulating evidence on its effectiveness. However, guidelines for optimal methods of simulation training do not currently exist. METHODS: Systematic reviews of the literature on 16 identified key questions were conducted and expert panel consensus recommendations determined using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology. OBJECTIVE: These evidence-based guidelines from the Society for Simulation in Healthcare intend to support healthcare professionals in decisions on the most effective methods for simulation training in healthcare. RESULTS: Twenty recommendations on 16 questions were determined using GRADE. Four expert recommendations were also provided. CONCLUSIONS: The first evidence-based guidelines for simulation training are provided to guide instructors and learners on the most effective use of simulation in healthcare.

Simulation vs workplace-based assessment in resuscitation: a cross-specialty descriptive analysis and comparison.

Background: Simulation-based assessment can complement workplace-based assessment of rare or difficult to assess Entrustable Professional Activities (EPAs). We aimed to compare the use of simulation-based assessment for resuscitation-focused EPAs in three postgraduate medical training programs and describe faculty perceptions of simulation-based assessment. Methods: EPA assessment scores and setting (simulation or workplace) were extracted from 2017-2020 for internal medicine, emergency medicine, and surgical foundations residents at the transition to discipline and foundations of discipline stages. A questionnaire was distributed to clinical competency committee members. Results: Eleven percent of EPA assessments were simulation-based. The proportion of simulation-based assessment did not differ between programs but differed between transition (38%) and foundations (4%) stages within surgical foundations only. Entrustment scores differed between settings in emergency medicine at the transition level only (simulation: 4.82 ± 0.60 workplace: 3.74 ± 0.93). 70% of committee members (n=20) completed the questionnaire. Of those that use simulation-based assessment, 45% interpret them differently than workplace-based assessments. 73% and 100% trust simulation for high-stakes and low-stakes assessment, respectively. Conclusions: The proportion of simulation-based assessment for resuscitation focused EPAs did not differ between three postgraduate medical training programs. Interpretation of simulation-based assessment data between committee members was inconsistent. All respondents trust simulation-based assessment for low-stakes, and the majority for high-stakes assessment. These findings have practical implications for the integration simulation into programs of assessment.

Contexte: Pour les activités professionnelles confiables (APC) qui sont rarement observées ou difficiles à évaluer, une évaluation en séance de simulation peut compléter celle en milieu de travail. Nous avons comparé le recours à une évaluation en séance de simulation pour les APC axées sur la réanimation dans trois programmes de formation médicale postdoctorale et décrit les perceptions de membres du corps professoral à propos de cette modalité d'évaluation. Méthodes: Nous avons extrait les scores et le cadre (simulation ou lieu de travail) d'évaluation des APC de 2017 à 2020 pour les résidents en médecine interne, en médecine d'urgence et en fondements chirurgicaux aux étapes de transition vers la discipline et de fondements de la discipline. Un questionnaire a été distribué aux membres des comités des compétences cliniques. Résultats: Onze pour cent des évaluations d'APC étaient faites lors de séances de simulation. Cette proportion était la même pour tous les programmes, mais dans le cadre des fondements chirurgicaux elle était différente selon qu'il s'agissait de l'étape de transition (38 %) ou de celle des fondements (4 %). Les scores de confiance différaient selon le cadre de l'évaluation uniquement pour les résidents en médecine d'urgence à l'étape de la transition (simulation : 4,82 ± 0,60; lieu de travail : 3,74 ± 0,93). Le questionnaire a été rempli par 70 % des membres des comités (n=20). Parmi ceux qui avaient eu recours à une évaluation en séance de simulation, 45 % avaient interprété les données de l'évaluation différemment de la façon dont ils interprètent les données d'évaluation en milieu de travail. Soixante-treize pour cent et 100 % d'entre eux font confiance à la simulation pour les évaluations à enjeux élevés et à faibles enjeux, respectivement. Conclusions: La proportion d'évaluations en séance de simulation pour les APC axées sur la réanimation était la même dans trois programmes de formation médicale postdoctorale. Les membres des comités de compétences cliniques n'ont pas interprété les données de ce type d'évaluation de manière uniforme. Tous les répondants font confiance à l'évaluation en séance de simulation pour les évaluations à faibles enjeux, et la plupart d'entre eux pour les évaluations à enjeux élevés. Ces données ont des implications pratiques pour l'intégration de la simulation dans les programmes d'évaluation.

Are we talking about practice? A randomized study comparing simulation-based deliberate practice and mastery learning to self-guided practice.

OBJECTIVES: Simulation-based technical skills training is now ubiquitous in medicine, particularly for high acuity, low occurrence (HALO) procedures. Mastery learning and deliberate practice (ML + DP) are potentially valuable educational methods, however, they are resource intensive. We sought to compare the effect of deliberate practice and mastery learning versus self-guided practice on skill performance of the rare, life-saving procedure, a bougie-assisted cricothyroidotomy (BAC). METHODS: We conducted a multi-center, randomized study at five North American emergency medicine (EM) residency programs. We randomly assigned 176 EM residents to either the ML + DP or self-guided practice groups. Three blinded airway experts independently evaluated BAC skill performance by video review before (pre-test), after (post-test) and 6-12 months (retention) after the training session. The primary outcome was post-test skill performance using a global rating score (GRS). Secondary outcomes included performance time and skill performance at the retention test. RESULTS: Immediately following training, GRS scores were significantly higher as mean performance improved from pre-test, (22, 95% CI = 21-23) to post-test (27, 95% CI = 26-28), (p < 0.001) for all participants. However, there was no difference between the groups on GRS scores (p = 0.2) at the post-test or at the retention test (p = 0.2). At the retention test, participants in the ML + DP group had faster performance times (66 s, 95% CI = 57-74) compared to the self-guided group (77 s, 95% CI = 67-86), (p < 0.01). CONCLUSIONS: There was no significant difference in skill performance between groups. Residents who received deliberate practice and mastery learning demonstrated an improvement in skill performance time.

ABSTRAIT: OBJECTIFS: La formation aux compétences techniques fondée sur la simulation est maintenant omniprésente en médecine, en particulier pour les procédures de grande acuité et de faible occurrence (HALO). L'apprentissage de la maîtrise et la pratique délibérée (ML+DP) sont des méthodes éducatives potentiellement précieuses, mais elles exigent beaucoup de ressources. Nous avons cherché à comparer l'effet de la pratique délibérée et de l'apprentissage de la maîtrise par rapport à la pratique autoguidée sur le rendement des compétences de la rare intervention de sauvetage, une cricothyroïdotomie à la bougie. MéTHODES: Nous avons mené une étude multicentrique randomisée dans le cadre de cinq programmes nord-américains de résidence en médecine d'urgence. Nous avons affecté au hasard 176 résidents en SE aux groupes de ML+DP ou de pratique autoguidée. Trois experts des voies respiratoires aveuglés ont évalué de façon indépendante la performance des compétences en BAC par examen vidéo avant (pré-test), après (post-test) et 6 à 12 mois (rétention) après la séance de formation. Le principal résultat a été le rendement des compétences après le test au moyen d'une cote globale (SRC). Les résultats secondaires comprenaient le temps consacré au rendement et le rendement des compétences au test de rétention. RéSULTATS: Immédiatement après la formation, les résultats des SRC étaient beaucoup plus élevés, car le rendement moyen s'est amélioré entre le prétest (22, IC à 95 % = 21 à 23) et le post-test (27, IC à 95 % = 26 à 28), (p < 0,001) pour tous les participants. Cependant, il n'y avait aucune différence entre les groupes sur les scores GRS (p = 0,2) au post-test ou au test de rétention (p = 0,2). Au test de rétention, les participants du groupe ML+DP avaient des temps de performance plus rapides (66 secondes, IC à 95 % = 57 à 74) que ceux du groupe autoguidé (77 secondes, IC à 95 % = 67 à 86) (p < 0,01). CONCLUSIONS: Il n'y avait pas de différence significative dans le rendement des compétences entre les groupes. Les résidents qui ont bénéficié d'une pratique délibérée et d'un apprentissage de la maîtrise ont démontré une amélioration du temps consacré aux compétences.

The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting.

Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.

Navigating gaps in practice: consensus recommendations for educational and support structures for emergency physicians returning from gaps in practice.

OBJECTIVE: The purpose of this study was to identify the learning needs of emergency physicians returning to Emergency Medicine (EM) practice after clinical leaves of less than 2 years, summarize existing return to practice programs, and propose recommendations regarding ideal educational and support structures for these physicians both during their practice gaps and upon return to EM. METHODS: A multiple-phased study was conducted to establish recommendations regarding ideal educational and support structures for emergency physicians returning from practice gaps of less than 2 years. The overall design involved an initial environmental scan of existing and exemplar programs and regulatory body positions, followed by interviews with EM Department Heads from across Canada, and then subsequent content analysis and recommendation derivation by EM medical education expert group consensus. These summary recommendations were further revised by consensus at the 2022 CAEP conference academic symposium to derive a final set of consensus recommendations. CONCLUSIONS: We have developed a set of recommendations regarding ideal educational and support structures for physicians experiencing gaps in practice of less than 2 years. This set of recommendations was informed by review of existing and exemplar programs, policies and experience of regulatory bodies, interviews with EM Department Heads across Canada, and a subsequent process of consensus at the 2022 CAEP conference academic symposium. It is hoped that this set of recommendations will inform discussions and potential strategies employed by departments to facilitate the smooth and effective return to EM practice for individuals experiencing gaps.

ABSTRAIT: OBJECTIFS: Le but de cette étude était de déterminer les besoins d'apprentissage des médecins d'urgence qui retournent à la pratique de la médecine d'urgence (ME) après des congés cliniques de moins de deux ans, de résumer les programmes de retour à la pratique existants. et de proposer des recommandations concernant les structures de formation et de soutien idéales pour ces médecins, à la fois pendant leurs lacunes dans la pratique et à leur retour à la GU. MéTHODES: Une étude en plusieurs phases a été menée afin d'établir des recommandations concernant les structures de formation et de soutien idéales pour les médecins d'urgence qui reviennent de lacunes de moins de deux ans. La conception globale comprenait une première analyse de l'environnement des programmes existants et exemplaires et des postes d'organismes de réglementation, suivie d'entrevues avec les chefs des services de GU de partout au Canada. et ensuite l'analyse du contenu et la formulation de recommandations par consensus du groupe d'experts en éducation médicale de la SE. Ces recommandations résumées ont été révisées par consensus lors du symposium universitaire de la conférence 2022 de l'ACMU afin d'en arriver à une série finale de recommandations consensuelles. CONCLUSION: Nous avons élaboré une série de recommandations concernant les structures de formation et de soutien idéales pour les médecins qui connaissent des lacunes dans la pratique depuis moins de deux ans. Cette série de recommandations a été éclairée par l'examen des programmes, des politiques et de l'expérience des organismes de réglementation existants et exemplaires, des entrevues avec les chefs des services de GU partout au Canada et un processus subséquent de consensus au symposium universitaire de la conférence 2022 de la CAEP. On espère que cette série de recommandations éclairera les discussions et les stratégies potentielles employées par les ministères pour faciliter le retour en douceur et efficace à la pratique de GU pour les personnes qui connaissent des lacunes.

Catalytic effect of multisource feedback for trauma team captains: a mixed-methods prospective study.

OBJECTIVES: To evaluate the impact and feasibility of multisource feedback compared with traditional feedback for trauma team captains (TTCs). DESIGN: A mixed-methods, non-randomised prospective study. SETTING: A level one trauma centre in Ontario, Canada. PARTICIPANTS: Postgraduate medical residents in emergency medicine and general surgery participating as TTCs. Selection was based on a convenience sampling method. INTERVENTION: Postgraduate medical residents participating as TTCs received either multisource feedback or standard feedback following trauma cases. MAIN OUTCOME MEASURES: TTCs completed questionnaires designed to measure the self-reported intention to change practice (catalytic effect), immediately following a trauma case and 3 weeks later. Secondary outcomes included measures of perceived benefit, acceptability, and feasibility from TTCs and other trauma team members. RESULTS: Data were collected following 24 trauma team activations: TTCs from 12 activations received multisource feedback and 12 received standard feedback. The self-reported intention for practice change was not significantly different between groups initially (4.0 vs 4.0, p=0.57) and at 3 weeks (4.0 vs 3.0, p=0.25). Multisource feedback was perceived to be helpful and superior to the existing feedback process. Feasibility was identified as a challenge. CONCLUSIONS: The self-reported intention for practice change was no different for TTCs who received multisource feedback and those who received standard feedback. Multisource feedback was favourably received by trauma team members, and TTCs perceived multisource feedback as useful for their development.

The Reliability of the Resuscitation Assessment Tool (RAT) in Assessing Emergency Medicine Resident Competence in Pediatric Resuscitation Scenarios: A Prospective Observational Pilot Study.

Introduction Emergency medicine (EM) postgraduate medical education in Canada has transitioned from traditional time-based training to competency-based medical education (CBME). In order to promote residents through stages of training, simulated assessments are needed to evaluate residents in high-stakes but low-frequency medical emergencies. There remains a gap in the literature pertaining to the use of evaluative tools in simulation, such as the Resuscitation Assessment Tool (RAT) in the new CBME curriculum design. Methods We completed a pilot study of resident physicians in one Canadian EM training program to evaluate the effectiveness and reliability of a simulation-based RAT for pediatric resuscitation. We recorded 10 EM trainees completing simulated scenarios and had nine EM physicians use the RAT tool to evaluate their performances. Generalizability theory was used to evaluate the reliability of the RAT tool. Results The mean RAT score for the management of pediatric myocarditis, cardiac arrest, and septic shock (appendicitis) across raters was 3.70, 3.73, and 4.50, respectively. The overall generalizability coefficient for testing simulated pediatric performance competency was 0.77 for internal consistency and 0.75 for absolute agreement. The performance of senior participants was superior to that of junior participants in the management of pediatric myocarditis (p = 0.01) but not statistically significant in the management of pediatric septic shock (p=0.77) or cardiac arrest (p =0.61). Conclusion Overall, our findings suggest that with an appropriately chosen simulated scenario, the RAT tool can be used effectively for the simulation of high-stakes and low-frequency scenarios for practice to enhance the new CBME curriculum in emergency medicine training programs.

Global health competencies in postgraduate medical education: a scoping review and mapping to the CanMEDS physician competency framework.

Background: Global Health opportunities are popular, with many reported benefits. There is a need however, to identify and situate Global Health competencies within postgraduate medical education. We sought to identify and map Global Health competencies to the CanMEDS framework to assess the degree of equivalency and uniqueness between them. Methods: JBI scoping review methodology was utilized to identify relevant papers searching MEDLINE, Embase, and Web of Science. Studies were reviewed independently by two of three researchers according to pre-determined eligibility criteria. Included studies identified competencies in Global Health training at the postgraduate medicine level, which were then mapped to the CanMEDS framework. Results: A total of 19 articles met criteria for inclusion (17 from literature search and two from manual reference review). We identified 36 Global Health competencies; the majority (23) aligned with CanMEDS competencies within the framework. Ten were mapped to CanMEDS roles but lacked specific key or enabling competencies, while three did not fit within the specific CanMEDS roles. Conclusions: We mapped the identified Global Health competencies, finding broad coverage of required CanMEDS competencies. We identified additional competencies for CanMEDS committee consideration and discuss the benefits of their inclusion in future physician competency frameworks.

Contexte: Les opportunités de santé mondiale sont populaires, avec de nombreux avantages rapportés. Il est toutefois nécessaire d'identifier et de situer les compétences en santé mondiale dans la formation médicale postdoctorale. Nous avons cherché à identifier et à mapper les compétences en santé mondiale au cadre le référentiel CanMEDS d'évaluer le degré d'équivalence et d'unicité entre elles. Méthodologie: La méthodologie de revue exploratoire de JBI a été utilisée pour identifier les articles pertinents qui recherchent MEDLINE, Embase et Web of Science. Les études ont été examinées indépendamment par deux des trois chercheurs selon des critères d'admissibilité prédéterminés. Les études incluses ont permis d'identifier les compétences dans la formation en santé mondiale au niveau de la médecine postdoctorale, qui ont ensuite été mises en correspondance avec le cadre le référentiel CanMEDS. Résultats: Au total, 19 articles répondaient aux critères d'inclusion (17 provenant d'une recherche documentaire et 2 d'un examen manuel des références). Nous avons identifié 36 compétences en santé mondiale; la majorité (23) correspondait aux compétences CanMEDS dans le cadre. Dix d'entre eux ont été mappés à des rôles canMEDS, mais n'avaient pas de compétences clés ou habilitantes précises, tandis que trois ne correspondaient pas aux rôles spécifiques de CanMEDS. Conclusions: Nous avons cartographié les compétences en santé mondiale identifiées, en trouvant une large couverture des compétences CanMEDS requises. Nous avons identifié d'autres compétences à examiner par le comité CanMEDS et nous discutons des avantages de leur inclusion dans les futurs cadres de compétences des médecins.

Emerging concepts in the CanMEDS physician competency framework.

Background: The CanMEDS physician competency framework will be updated in 2025. The revision occurs during a time of disruption and transformation to society, healthcare, and medical education caused by the COVID-19 pandemic and growing acknowledgement of the impacts of colonialism, systemic discrimination, climate change, and emerging technologies on healthcare and training. To inform this revision, we sought to identify emerging concepts in the literature related to physician competencies. Methods: Emerging concepts were defined as ideas discussed in the literature related to the roles and competencies of physicians that are absent or underrepresented in the 2015 CanMEDS framework. We conducted a literature scan, title and abstract review, and thematic analysis to identify emerging concepts. Metadata for all articles published in five medical education journals between October 1, 2018 and October 1, 2021 were extracted. Fifteen authors performed a title and abstract review to identify and label underrepresented concepts. Two authors thematically analyzed the results to identify emerging concepts. A member check was conducted. Results: 1017 of 4973 (20.5%) of the included articles discussed an emerging concept. The thematic analysis identified ten themes: Equity, Diversity, Inclusion, and Social Justice; Anti-racism; Physician Humanism; Data-Informed Medicine; Complex Adaptive Systems; Clinical Learning Environment; Virtual Care; Clinical Reasoning; Adaptive Expertise; and Planetary Health. All themes were endorsed by the authorship team as emerging concepts. Conclusion: This literature scan identified ten emerging concepts to inform the 2025 revision of the CanMEDS physician competency framework. Open publication of this work will promote greater transparency in the revision process and support an ongoing dialogue on physician competence. Writing groups have been recruited to elaborate on each of the emerging concepts and how they could be further incorporated into CanMEDS 2025.

Contexte: Le référentiel de compétences CanMEDS pour les médecins sera mis à jour en 2025. Cette révision arrive à un moment où la société, les soins de santé et l'enseignement médical sont bouleversés et en pleine mutation à cause de la pandémie de la COVID-19. On est aussi à l'heure où l'on reconnaît de plus en plus les effets du colonialisme, de la discrimination systémique, des changements climatiques et des nouvelles technologies sur les soins de santé et la formation des médecins. Pour effectuer cette révision, nous avons avons extrait de la littérature scientifique les concepts émergents se rapportant aux compétences des médecins. Méthodes: Les concepts émergents ont été définis comme des idées ayant trait aux rôles et aux compétences des médecins qui sont débattues dans la littérature, mais qui sont absentes ou sous-représentées dans le cadre CanMEDS 2015. Nous avons réalisé une recherche documentaire, un examen des titres et des résumés, et une analyse thématique pour repérer les concepts émergents. Les métadonnées de tous les articles publiés dans cinq revues d'éducation médicale entre le 1er octobre 2018 et le 1er octobre 2021 ont été extraites. Quinze auteurs ont effectué un examen des titres et des résumés pour relever et étiqueter les concepts sous-représentés. Deux auteurs ont procédé à une analyse thématique des résultats pour dégager les concepts émergents. Une vérification a été faite par les membres de l'équipe. Résultats: Parmi les 4973 articles dépouillés, 1017 (20,5 %) abordaient un concept émergent. Les dix thèmes suivants sont ressortis de l'analyse thématique: l'équité, la diversité, l'inclusion et la justice sociale; l'antiracisme; l'humanité du médecin; la médecine fondée sur les données; les systèmes adaptatifs complexes; l'environnement de l'apprentissage clinique; les soins virtuels; le raisonnement clinique; l'expertise adaptative; et la santé planétaire. L'ensemble de ces thèmes ont été approuvés comme concepts émergents par l'équipe de rédaction. Conclusion: Cet examen de la littérature a permis de relever dix concepts émergents qui peuvent servir à éclairer la révision du référentiel de compétences CanMEDS pour les médecins qui aura lieu en 2025. La publication en libre accès de ce travail favorisera la transparence du processus de révision et le dialogue continu sur les compétences des médecins. Des groupes de rédaction ont été recrutés pour développer chacun des concepts émergents et pour examiner la façon dont ils pourraient être intégrés dans la version du référentiel CanMEDS de 2025.

Lessons learned and new strategies for success: Evaluating the Implementation of Competency-Based Medical Education in Queen's Pediatrics.

Objectives: In 2017, Queen's University launched Competency-Based Medical Education (CBME) across 29 programs simultaneously. Two years post-implementation, we asked key stakeholders (faculty, residents, and program leaders) within the Pediatrics program for their perspectives on and experiences with CBME so far. Methods: Program leadership explicitly described the intended outcomes of implementing CBME. Focus groups and interviews were conducted with all stakeholders to describe the enacted implementation. The intended versus enacted implementations were compared to provide insight into needed adaptations for program improvement. Results: Overall, stakeholders saw value in the concept of CBME. Residents felt they received more specific feedback and monthly Competence Committee (CC) meetings and Academic Advisors were helpful. Conversely, all stakeholders noted the increased expectations had led to a feeling of assessment fatigue. Faculty noted that direct observation and not knowing a resident's previous performance information was challenging. Residents wanted to see faculty initiate assessments and improved transparency around progress and promotion decisions. Discussion: The results provided insight into how well the intended outcomes had been achieved as well as areas for improvement. Proposed adaptations included a need for increased direct observation and exploration of faculty accessing residents' previous performance information. Education was provided on the performance expectations of residents and how progress and promotion decisions are made. As well, "flex blocks" were created to help residents customize their training experience to meet their learning needs. The results of this study can be used to inform and guide implementation and adaptations in other programs and institutions.

Gender Differences in Emergency Medicine Attending Physician Comments to Residents: A Qualitative Analysis.

Importance: Prior studies have revealed gender differences in the milestone and clinical competency committee assessment of emergency medicine (EM) residents. Objective: To explore gender disparities and the reasons for such disparities in the narrative comments from EM attending physicians to EM residents. Design, Setting, and Participants: This multicenter qualitative analysis examined 10 488 narrative comments among EM faculty and EM residents between 2015 to 2018 in 5 EM training programs in the US. Data were analyzed from 2019 to 2021. Main Outcomes and Measures: Differences in narrative comments by gender and study site. Qualitative analysis included deidentification and iterative coding of the data set using an axial coding approach, with double coding of 20% of the comments at random to assess intercoder reliability (κ, 0.84). The authors reviewed the unmasked coded data set to identify emerging themes. Summary statistics were calculated for the number of narrative comments and their coded themes by gender and study site. χ2 tests were used to determine differences in the proportion of narrative comments by gender of faculty and resident. Results: In this study of 283 EM residents, of whom 113 (40%) identified as women, and 277 EM attending physicians, of whom 95 (34%) identified as women, there were notable gender differences in the content of the narrative comments from faculty to residents. Men faculty, compared with women faculty, were more likely to provide either nonspecific comments (115 of 182 [63.2%] vs 40 of 95 [42.1%]), or no comments (3387 of 10 496 [32.3%] vs 1169 of 4548 [25.7%]; P < .001) to men and women residents. Compared with men residents, more women residents were told that they were performing below level by men and women faculty (36 of 113 [31.9%] vs 43 of 170 [25.3%]), with the most common theme including lack of confidence with procedural skills. Conclusions and Relevance: In this qualitative study of narrative comments provided by EM attending physicians to residents, multiple modifiable contributors to gender disparities in assessment were identified, including the presence, content, and specificity of comments. Among women residents, procedural competency was associated with being conflated with procedural confidence. These findings can inform interventions to improve parity in assessment across graduate medical education.